A wide variety of data is collected or captured in electronic media. Unfortunately, the practical use of that data is often limited by the data's native stored format. That is, the ability to consume or take action based on captured electronic data is determined by another service's ability to recognize or properly translate the data's format into other formats which the service can process. In many instances captured electronic data exists in proprietary or unique data formats which are produced by another service as output. Thus, if there is a desire to integrate the captured electronic data with consuming services, the data format of the captured electronic data must be translated for the consuming services.
This particular problem is most noticeable with captured log information associated with a processing service. Log files are rich with information about activities of a processing service (e.g., identity information, time information, resource information, task performance information, etc.) and events (e.g., failures, access attempts, etc.). Moreover, log files are often manually inspected or automatically inspected with batch processing services that are designed to recognize and process the data formats of specific log files.
Yet, batch processing is not an optimal solution because separate batch services are needed for each unique log file data format. This means that a myriad of batch services can rapidly proliferate within an enterprise, with each requiring maintenance and support. Additionally, if a particular batch service is designed to interface with another service based on evaluation of a log file's contents that batch service will need to be modified if a different service requires integration with the log file's contents. Accordingly, upgrades and enhancements are regularly made to batch files as new services are integrated with a log file's contents.
The problem is not limited to log files, because there are a variety of situations where an enterprise could profitably benefit from automatically harvesting and integrating output from one service with another service. For example, a financial service's (e.g., banking transactions, stock transactions, etc.) output could be automatically consumed by an alerting service (e.g., instant messaging (IM), electronic mail (email), etc.) in situations where the output is deemed significant enough to warrant some processing by the alerting service.
In many instances, the significance of output is determined when it is associated with an event, and based on the presence of that event some action is desired by another service (e.g., alerting service emails a business auditor). However, in many circumstances, a single event and its associated data from a data stream may depend upon other events and their data included elsewhere within the data stream. Thus, there is a need not only to translate events to different data formats but to also package or group detected events within data streams for integration with consuming services.
Accordingly, techniques are needed for improved serialization of events detected within data streams.